System, method, and program for distributing video

ABSTRACT

A video distribution system distributes a video including animation of a character object generated based on a motion of an actor to client devices used by a plurality of viewing users. The video distribution system includes: one or more computer processors; a storage that stores objects associated with each of the plurality of viewing users; and a display device seeable by the actor. The one or more computer processors execute computer-readable instructions to: receive a display request for any of the objects from a first viewing user among the plurality of viewing users; and display object information on the display device, the object information including information on the object corresponding to the display request in association with identification information of the first viewing user.

TECHNICAL FIELD

The present invention relates to a system, method, and program fordistributing a video.

BACKGROUND

Some video distribution systems have been known to generate animation ofa character object based on an actor's motions and distribute a videoincluding the generated animation (see, for example, Patent Literature1).

It is proposed that such video distribution systems distribute a videoto multiple viewing users and the video contain a desired objectdisplayed in accordance with operation by the viewing users. Forexample, the viewing users are capable of having a gift object theypresented to the character object be displayed in the video.

RELEVANT REFERENCES Patent Literature

Patent Literature 1: Japanese Patent Application Publication No.2015-184689

SUMMARY

To increase the number of viewing users who view the distributed videoand the number of times the distributed video is viewed, the characterobject should have originality, and in addition, it is important toincrease the chances for communication between the actor and the viewingusers through the character object. However, the actor has to providemotions to the character object, utter a voice as necessary, andmoreover, react to a gift object. To increase the chances forcommunication with the viewing users, the actor has to pay additionalattention. Therefore, it is actually difficult for the actor to increasethe chances for communication with the viewing users.

One object of the present disclosure is to provide a mechanism forincreasing the chances for communication with the viewing users.

According to one aspect of the present disclosure, provided is a videodistribution system for distributing a video including animation of acharacter object generated based on a motion of an actor to clientdevices used by a plurality of viewing users. The video distributionsystem comprises: one or more computer processors; a storage that storesobjects associated with each of the plurality of viewing users; and adisplay device seeable by the actor. The one or more computer processorsexecute computer-readable instructions to: receive a display request forany of the objects from a first viewing user among the plurality ofviewing users; and display object information on the display device, theobject information including information on the object corresponding tothe display request in association with identification information ofthe first viewing user.

According to another aspect, provided is a video distribution methodperformed by one or more computer processors executing computer-readableinstructions to distribute a video including animation of a characterobject generated based on a motion of an actor to client devices used bya plurality of viewing users. The video distribution method comprises:storing, on a storage, objects associated with each of the plurality ofviewing users; and receive a display request for any of the objects froma first viewing user among the plurality of viewing users; and displayobject information on a display device disposed at a position seeable bythe actor, the object information including information on the objectcorresponding to the display request in association with identificationinformation of the first viewing user.

According to still another aspect, provided is a video distributionprogram for distributing a video including animation of a characterobject generated based on a motion of an actor to client devices used bya plurality of viewing users. The video distribution program causes oneor more computer processors to: storing, on a storage, objectsassociated with each of the plurality of viewing users; and receive adisplay request for any of the object from a first viewing user amongthe plurality of viewing users; and display object information on adisplay device disposed at a position seeable by the actor, the objectinformation including information on the object corresponding to thedisplay request in association with identification information of thefirst viewing user.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic diagram of a video distribution system accordingto one embodiment.

FIG. 2 is a schematic diagram of installations of a studio where a videois produced for distribution by the video distribution system of FIG. 1.

FIG. 3 is a block diagram illustrating a part of the video distributionsystem of FIG. 1.

FIG. 4 is a block diagram illustrating a client device of FIG. 1.

FIG. 5 illustrates an example of user management data.

FIG. 6 illustrates an example of a possession list.

FIG. 7 illustrates an example of a candidate list.

FIG. 8 illustrates examples of post messages.

FIG. 9 illustrates an example of a video displayed on the client deviceof FIG. 4.

FIG. 10 illustrates an example of a video displayed on the client deviceof FIG. 4.

FIG. 11 illustrates an example of a video displayed on the client deviceof FIG. 4.

FIG. 12 illustrates an example of a video displayed on the client deviceof FIG. 4.

FIG. 13 illustrates an example of a view displayed on a display seen byan actor.

FIG. 14 illustrates an example of a view displayed on a display seen byan actor.

FIG. 15 illustrates an example of a view displayed on a display seen byan actor.

FIG. 16 is a flowchart of a video distribution process in oneembodiment.

FIG. 17 is a flowchart of a process for receiving a decorative object inone embodiment.

FIG. 18 is a flowchart of a process for displaying a decorative objectin one embodiment.

FIG. 19 is a flowchart of a process for displaying a post message in oneembodiment.

DESCRIPTION OF THE EMBODIMENTS

With reference to the appended drawings, the following describes a videodistribution system according to one embodiment of the presentdisclosure. As shown in FIG. 1, the video distribution system 1 includesclient devices 10, a server 20, and a studio unit 30. The client devices10, the server 20, and the studio unit 30 are communicativelyinterconnected over a network 11. The network 11 may include variousnetworks such as a local area network and the Internet.

In the video distribution system 1, a distributor uses the studio unit30 to produce a video having contents including animation of a characterobject based on motions of an actor. The character object is anindividual of a virtual human, animal, or the like. The video producedis distributed via the server to the client devices 10 used by viewingusers. The distributor may be an actor, an administrator of the studiounit 30, or others.

The studio unit 30 will be now outlined with reference to FIG. 2. Thestudio unit 30 includes a tracking system 31, a controller 40, a maindisplay 41, a sub display 43 (display device), a microphone 44 (see FIG.3), and a supporter computer 45.

The tracking system 31 obtains information on the motions of the actorA. The actor A is, for example, a human, animal, or moving object thatgives a performance. The actor A may be, for example, an autonomousrobot. A studio room R may contain a plurality of actors.

The tracking system 31 senses the position of the actor A. The trackingsystem 31 may also sense the facial expression of the actor A, theposition of a virtual camera set in a virtual space, and the like. Forexample, the tracking system 31 includes external sensors 32, wearablesensors 33, and a positioning device 35. These are based on theoutside-in method for sensing at least one of the orientations and thepositions of the wearable sensors 33 attached to the actor A.

The wearable sensors 33 are attached to the actor A. A plurality ofsensors 33 are attached to the actor A. For example, six wearablesensors 33 are attached to the left wrist, the right wrist, the leftinstep, the right instep, the hip, and the top of the head of the actorA, respectively. The wearable sensors 33 may be attached to the actor Avia an attachment, or the actor A may wear a costume having the wearablesensors 33 attached thereto. The wearable sensors 33 may be attached tovarious body parts of the actor A. For example, the wearable sensors 33may be attached to fingers to sense motions of the fingers. The numberof wearable sensors 33 attached to the actor A may be less than or morethan six.

The wearable sensors 33 output information for identifying theorientation and position thereof to the positioning device 35 incooperation with the external sensors 32. One example of the externalsensors 32 in the outside-in method is multi-axis laser emitters. Theexternal sensors 32 emit a pulsed laser beam for synchronization. Thewearable sensors 33 include a sensor for sensing a laser beam, and theysense the position and orientation thereof while synchronizing with thesynchronization pulse. Examples of the wearable sensors 33 and theexternal sensors 32 include Vive Tracker™ and Vive Base Station from HTCCorporation™. Additionally or alternatively, the wearable sensors 33 mayinclude a built-in sensor for identifying the orientation or positionthereof autonomously. For example, the wearable sensors 33 may includeat least one of a multi-axis acceleration sensor, a gyroscope, and amagnetic field sensor. It is also possible to combinedly use thewearable sensors 33 that cooperate with the external sensors 32 and thewearable sensors 33 that do not cooperate with the external sensors 32.Further, the wearable sensors 33 may be capable of sensing theorientation or position thereof alone, that is, without cooperating withthe external sensors.

The tracking system 31 includes a viewpoint sensor 36 and a camera 37.The position of the viewpoint sensor 36 can be changed by the user. Theviewpoint sensor 36 is configured in the same manner as the wearablesensors 33. The viewpoint sensor 36 outputs information for identifyingthe orientation and position thereof to the positioning device 35wirelessly or via a communication cable. The viewpoint sensor 36 is setas a viewpoint of a virtual camera in a virtual space. The viewpointsensor 36 may be supported, for example, by a stand or by a gimbal orstabilizer. The gimbal may be shaped to be graspable by an actor orother persons. By supporting the viewpoint sensor 36 with the gimbal orstabilizer, it is possible to suppress camera shake (shake in video).

The camera 37 is disposed so as to capture an image of the face of theactor A. For example, the camera 37 is attached to the head or neck ofthe actor A or positioned in front of the actor A. The camera 37continuously captures images of the face of the actor A to obtainimaging data of the face of the actor A, and sends the imaging data tothe server 20. The camera 37 may be a 3D camera capable of sensing thedepth of a face of a person.

The microphone 44 is disposed at such a position that it can collect thevoice uttered by the actor A. For example, the microphone 44 is attachedto the head or neck of the actor A or positioned in front of the actorA. The microphone 44 sends signals to the server 20 in accordance withthe voice.

The controller 40 is operated by the actor A. The controller 40 outputssignals to the server 20 in accordance with the operation of the actorA. The controller 40 may include a built-in sensor for identifying theorientation or position thereof autonomously. For example, thecontroller 40 may include at least one of a multi-axis accelerationsensor, a gyroscope, and a magnetic field sensor. Additionally oralternatively, the controller 40 may be capable of sensing the positionin cooperation with the external sensors 32.

The main display 41 is configured to display a video generated by theserver 20. A single main display 41 may be provided, or as shown, aplurality of main displays 41 may be provided. The video displayed onthe main display 41 includes animation of the character object.

The sub display 43 is configured to display information received fromthe server 20 or the supporter computer 45. A single sub display 43 maybe provided, or a plurality of sub displays 43 may be provided. Theinformation sent from the server 20 or the supporter computer 45 to themain display 41 may include, for example, text information, imageinformation, and various other information. The main display 41 and thesub display 43 are disposed at such positions as to be seen by the actorA.

The supporter computer 45 is used by one or more operators (hereinreferred to as “the supporters”) in a room adjacent to the studio room.Since the room in which the supporter computer 45 is installed isseparated from the studio room R by the glass window, the supportersusing the supporter computer 45 can see the actor A. In the embodimentshown, two supporters B1, B2 are in this room. The supporter computer 45outputs various information to the sub display 43 in accordance with theoperation of the supporters. In this specification, the supporter B1 andthe supporter B2 may be collectively referred to as the “supporter” whenit is not necessary to distinguish them from each other. The componentsand functions of the studio unit 30 shown in FIG. 2 are merely anexample. The studio unit 30 applicable to the invention may includevarious components that are not shown. For example, the studio unit 30may include a projector. The projector is able to project a videodistributed to the client devices 10 or other client devices on ascreen.

The server 20 will be now described with reference to FIG. 3. The server20 is configured to distribute to the viewing users a video thatincludes animation of a character object based on the motions of theactor. The character object included in the video may bemotion-controlled in a virtual space. The server 20 includes a computerprocessor 21, a communication interface (I/F) 22, and a storage 23(storage medium).

The computer processor 21 is a computing device which loads variousprograms realizing an operating system and various functions from thestorage 23 or any other storage into a memory unit and executesinstructions included in the loaded programs. The computer processor 21is, for example, a CPU, an MPU, a DSP, a GPU, any other computingdevice, or a combination thereof. At least a part of the processesperformed by the computer processor 21 may be realized by means of acircuit such as ASIC, PLD, FPGA, MCU, or the like. Although the computerprocessor 21 is illustrated as a single component in FIG. 3, thecomputer processor 21 may be a collection of a plurality of physicallyseparate computer processors. In this specification, a program orinstructions included in the program that are described as beingexecuted by the computer processor 21 may be executed by a singlecomputer processor or executed by a plurality of computer processorsdistributively. Further, a program or instructions included in theprogram executed by the computer processor 21 may be executed by one ormore virtual computer processors.

The computer processor 21 functions as a motion data processing unit21A, a video generation unit 21B, a video distribution unit 21C, a userrequest processing unit 21D, a user information management unit 21E, anda purchase processing unit 21F by executing computer-readableinstructions included in the distribution program stored on the storage23 or any other storage. At least some of the functions that can berealized by the computer processor 21 may be realized by a computerprocessor other than the computer processor 21 of the video distributionsystem 1. For example, at least some of the functions realized by thecomputer processor 21 may be realized by a computer processor of acomputer included in the studio unit 30.

The motion data processing unit 21A generates body motion data based ontracking information obtained by the wearable sensors 33 attached to theactor. The body motion data is a time-series digital representation ofat least one of the positions and orientations of body parts of theactor. The body motion data is serially generated with time as needed.For example, the body motion data may be generated at predeterminedsampling time intervals.

The motion data processing unit 21A also generates face motion data,which is a time-series digital representation of motions of the face ofthe actor, based on captured image data of the camera 37 included in thestudio unit 30. The face motion data is serially generated with time asneeded. For example, the face motion data may be generated atpredetermined sampling time intervals.

The video generation unit 21B applies the body motion data and the facemotion data generated by the motion data processing unit 21A to a pieceof model data included in the model data 23A stored on the storage 23.The video generation unit 21B thus generates animation of a characterobject in which the character object moves in a virtual space andchanges its facial expression.

Further, the video generation unit 21B constructs a virtual space usingthe object data 23B stored on the storage 23, and generates a video thatincludes the image of the virtual space and the animation of thecharacter object corresponding to the actor A. The video generation unit21B may use the position of the viewpoint sensor 36 as the position ofthe virtual camera that is the viewpoint of the video. The relativeposition of the actor A relative to the position of the virtual camerais applied to the position of the character object in the virtual space.Accordingly, when the actor A changes the position or orientation of theviewpoint sensor 36 as desired, the character object in the generatedvideo is viewed from the left side thereof, viewed from the right sidethereof, viewed down from above, or viewed in other various ways. Inaddition, the video generation unit 21B may change the settinginformation (such as a focus and an angle of view) of the virtual camerabased on the position of the virtual camera.

If a viewing user is associated with picture data of an avatar of theviewing user, the video generation unit 21B may generate animation ofthe avatar of the viewing user in addition to the animation of thecharacter object. The avatar may be shown in the video so as to movebased on the operation of the viewing user on the client device 10.

The video distribution unit 21C sends to the client devices 10 a list ofvideos for distribution and distributes the videos generated by thevideo generation unit 21B. The video distribution unit 21C maydistribute in real time the video being taken in the studio room R ordistribute a previously finished video (recorded video) in accordancewith a distribution schedule or the like. The video distribution unit21C may be configured to be able to distribute a plurality of videos inthe same period of time. For example, the video distribution unit 21Cmay distribute the video of “character X” performed by an actor Al andthe video of “character Y” performed by an actor A2 from the samestarting time (e.g., “20:00”) or in overlapping time periods. Inresponse to a list display request from a client device 10, the videodistribution unit 21C sends to the client device 10 a list of videosthat can be delivered. When receiving a video delivery request from theclient device 10, the video distribution unit 21C sends the videoselected by the viewing user to the client device 10 based on the videodelivery request.

The user request processing unit 21D receives a display request for anobject and a display request for a post message from a client device 10of a viewing user, and performs processing in accordance with thedisplay requests. The viewing user is able to send a display request foran object to the server 20 by operating his/her client device 10. Theobject may be a gift object provided from the viewing user to thecontents. The gift object is a digital gift from the viewing user to theactor or the like. For the viewing user to request display of the giftobject, some or no price may be required to be paid. The viewing usermay pay a price when the gift object is displayed in response to thedisplay request. The gift object may be displayed in the virtual spacecontinuously. The gift object may be displayed in the virtual spacetemporarily for temporal staging in the virtual space. The displayrequest for a gift object may include the user ID of the viewing userand the identification information (object ID) of the object for whichthe display request is sent.

When receiving a display request for a gift object from a viewing user,the user request processing unit 21D determines whether or not the actoror the supporter needs to make a judgment as to the display of the giftobject. If the user request processing unit 21D determines that theactor or the supporter does not need to make a judgment as to therequested display of the gift object, the user request processing unit21D causes the video generation unit 21B to display in the virtual spacethe object for which display is requested.

On the other hand, if the user request processing unit 21D determinesthat the actor or the supporter needs to make a judgment as to therequested display of the gift object, the user request processing unit21D displays on the sub display 43 the gift object for which display isrequested. When the actor or the supporter gives an instruction todisplay the gift object for which display is requested, the user requestprocessing unit 21D causes the video generation unit 21B to display theobject for which display is requested.

The user request processing unit 21D also performs a process fordisplaying a post message received from the client device 10 in thevideo. The post message includes the identification information (user IDor terminal ID) of the viewing user and the posting date and time, inaddition to the content of the message.

When receiving the post message, the user request processing unit 21Ddetermines whether or not the post message can be displayed in thecurrent scene. If the user request processing unit 21D determines thatthe received post message can be displayed, the user request processingunit 21D causes the video generation unit 21B to display the postmessage in a predetermined region in the video. The user requestprocessing unit 21D also displays the post message and the attributeinformation of the post message on the sub display 43.

The user information management unit 21E obtains the viewing situationof the viewing user and records the viewing situation on the storage 23in association with the viewing user. The user information managementunit 21E records as a history the video being viewed by the viewing useron a viewing application, that is, the “room” of the actor “visited” bythe viewing user, in association with the user ID of the viewing user.The user information management unit 21E also groups the viewing usersin accordance with the viewing situation of each viewing user.

The purchase processing unit 21F receives from the client device 10 adisplay request for purchasable contents such as a gift object relatedto the video, upon operation of the viewing user. After receiving thedisplay request for purchasable contents, the purchase processing unit21F sends the information on the purchasable contents to the clientdevice 10. The information on the purchasable contents may include thetypes, images, and prices of the purchasable contents and variousinformation necessary for purchase. The viewing user refers to theinformation of the purchasable contents displayed on the client device10 and selects a desired purchasable content. When the viewing user hasselected a purchasable content, the client device 10 sends a purchaserequest for the purchasable content to the server 20. The purchaseprocessing unit 21F performs a payment process based on the purchaserequest. When the payment process is completed, the server 20 recordsthe purchased content as being possessed by the viewing user. Thepurchasable content may be other than the gift object, for example, anobject related to viewing of the video other than the gift object, aright to view the video at a different image quality, or a right to viewa two-dimensional image or a three-dimensional image. The purchasablecontent may also be a video itself. The sales of the purchasable giftobject and other contents are dispensed to the distributor. A part ofthe sales may be dispensed to the administrator of the studio unit 30.

The communication I/F 22 may be implemented as hardware, firmware, orcommunication software such as a TCP/IP driver, or a combinationthereof. The server 20 is able to send and receive data to and fromother devices via the communication I/F 22.

The storage 23 is a storage device accessed by the computer processor21. The storage 23 is, for example, a magnetic disk, an optical disk, asemiconductor memory, or various other storage devices capable ofstoring data. Various programs may be stored on the storage 23. At leastsome of the programs and various data that can be stored on the storage23 may be stored on a storage that is physically separated from theserver 20.

Next, a description is given of the positioning device 35 and thesupporter computer 45 of the studio unit 30. The positioning device 35includes a computer processor 35A, a storage 35B (storage medium), and acommunication interface (I/F) 35C. Similarly to the computer processor21 of the server 20, the computer processor 35A may be any computingdevice such as a CPU. Similarly to the storage 23 of the server 20, thestorage 35B may be any storage device such as a magnetic disk. Similarlyto the communication interface 22 of the server 20, the communicationinterface (I/F) 35C may be any driver, software, or combination thereoffor communicating with other devices. The computer processor 35Aexecutes a position detecting program stored on the storage 35B or anyother storage, thereby obtaining tracking information from the externalsensors 32 or the wearable sensors 33 and identifying the positions ofthe wearable sensors 33. Also, the computer processor 35A executes aposition detecting program stored on the storage 35B or any otherstorage, thereby obtaining tracking information from the externalsensors 32 or the viewpoint sensor 36 and identifying the position ofthe viewpoint sensor 36. The positioning device 35 sends the identifiedposition to the server 20.

The supporter computer 45 includes a computer processor 46, acommunication I/F 47, a storage 48 (storage medium), a display 49, andan input interface 50.

Similarly to the computer processor 21 of the server 20, the computerprocessor 46 may be any computing device such as a CPU. Similarly to thecommunication I/F 22, the communication I/F 47 may be, a driver,software, or a combination thereof for communicating with other devices.Similarly to the storage 23, the storage 48 may be a storage devicecapable of storing data such as a magnetic disk. The storage 48 storesvarious programs. The display 49 may be a liquid crystal display, anorganic EL display, an inorganic EL display or any other display devicecapable of displaying images. The input interface 50 may be any pointingdevice that receives input from the supporter such as a mouse and akeyboard.

The computer processor 46 functions as a display control unit 46A and aprogress management unit 46B by executing computer-readable instructionsincluded in a program stored on the storage 48 or the like. At leastsome of the functions that can be realized by the computer processor 46may be realized by a computer processor other than the computerprocessor 46 of the video distribution system 1. Also, at least some ofthe functions described herein to be realized by the computer processor46 may be realized by, for example, the computer processor 21 of theserver 20.

The display control unit 46A is configured to display on the maindisplay 41 or the sub display 43 such a view that can be seen by theactor but cannot be seen by the viewing user and add various informationto the video distributed to the viewing users, in accordance withvarious input operation by the supporter via the input interface 50.

The supporter computer 45 may be configured to be capable of changingthe settings of the components of the studio unit 30 in accordance withthe operation by the supporter. The supporter computer 45 can change,for example, the setting of the interval of scanning performed by theexternal sensors 32, the setting of the position or orientation of thewearable sensors 33, and various settings of other devices. Thesupporter is able to input a message to the supporter computer 45, andthe inputted message is displayed on the main display 41.

The client device will now be described with reference to FIG. 4. Theclient device 10 is an information processing device such as asmartphone. In addition to the smartphone, the client device 10 may be amobile phone, a tablet, a personal computer, an electronic book reader,a wearable computer including a head-mounted display, a game console, orany other information processing device that is capable of playingvideos.

The client device 10 includes a computer processor 12, a storage 13(storage medium), an input interface (I/F) 14, a communication interface(I/F) 15, and a display 16. Similarly to the computer processor 21 ofthe server 20, the computer processor 12 may be any computing devicesuch as a CPU. Similarly to the storage 23 of the server 20, the storage13 may be any storage device such as a magnetic disk. The storage 13stores an application program for viewing videos distributed from theserver 20. The input I/F 14 may be any input interface that receivesinput from the viewing user such as a touch panel. Similarly to thecommunication I/F 22, the communication I/F 15 maybe, a driver,software, or a combination thereof for communicating with other devices.The display 16 may be any display device capable of displaying images,such as an organic EL display or a liquid crystal display, that isintegrated with a touch panel, for example.

When receiving operation of the viewing user via the input I/F 14, theclient device 10 starts the viewing application program implemented onthe client device 10. The viewing application has at least a functionfor playing the videos, and it may have any other function additionally.The computer processor 12 executes computer-readable instructionsincluded in the viewing application program and obtains a list of videosfor distribution from the server 20. The client device 10 displays theobtained list on the display. The viewing user selects one of the videosincluded in the list. When receiving selection operation of the viewinguser, the client device 10 sends to the server 20 a video deliveryrequest for requesting delivery of the selected video. Alternatively,the client device 10 may be configured to start a web browser stored onthe storage 13, access the server 20 via the web browser, and display onthe display 16 a web page written in a markup language such as HMTL orXML. The web page contains the list of videos. Alternatively, the clientdevice 10 may display the list of videos by using both the viewingprogram and the web browser (in-application browser) stored on itsstorage 13.

When receiving the data of the video from the server 20, the clientdevice 10 displays the video on the display 16 based on the receiveddata. The viewing user is able to input a message regarding thedistributed video via the input I/F 14 of the client device 10 to postthe message to the server 20. The message posted from each viewing usermay be displayed superimposed on the video. In this way, interaction isaccomplished between the actor and the viewing users and between theviewing users.

Next, a description is given of various data stored on the storage 23 ofthe server 20. The storage 23 stores the model data 23A, the object data23B, user management data 23C, a possession list 23D, a candidate list23E, and a post message 23F. The storage 23 may also store the videobeing distributed or videos having been already distributed by the videodistribution system 1.

The model data 23A is model data for generating animation of acharacter. The model data 23A may be three-dimensional model data forgenerating three-dimensional animation, or it may be two-dimensionalmodel data for generating two-dimensional animation. The model data 23Aincludes, for example, rig data (also referred to as “skeleton data”)representing a skeleton of a character, and surface data representingthe shape or texture of a surface of the character. The model data 23Amay include two or more different pieces of model data. Each piece ofmodel data may either include different rig data or include the same rigdata. Each piece of model data may either include different surface dataor include the same surface data. In the illustrated embodiment, inorder to generate a character object corresponding to the actor, themodel data 23A includes at least two types of model data different fromeach other.

The object data 23B includes asset data used for constructing a virtualspace in the video. The object data 23B includes data for rendering abackground of the virtual space in the video, data for rendering variousobjects displayed in the video, and data for rendering any other objectsdisplayed in the video. The object data 23B may include object positioninformation indicating the position of an object in the virtual space.

In addition to the above, the object data 23B may include gift objectsdisplayed in the video in response to a display request from viewingusers of the client devices. The gift objects may include an effectobject, a decorative object, and a normal object. The viewing users maypossess some of the gift objects for a price and the others for noprice. Moreover, an upper limit may be set for the number of objectsthat a viewing user is allowed to purchase or the amount of money thatthe viewing user is allowed to spend for objects.

The effect object is an object that affects the impression of the entireview of the distributed video, and is, for example, an objectrepresenting confetti or an object representing a firework. The objectrepresenting confetti may be displayed on the entire view, which canchange the impression of the entire view. For example, the effect objectmay be represented by a particle system that represents an event formedby moving particles. In the particle system, a duration of display maybe set for each particle. The effect object may be either a staticobject or a dynamic object. The effect object may be displayed so as tooverlap with the character object, but unlike the decorative object, itis not displayed in association with a specific boy part of thecharacter object.

The decorative object is an object displayed in the view in associationwith a specific boy part of the character object. For example, thedecorative object may be displayed in the view such that it contactswith a specific body part of the character object with which it isassociated. The decorative object may be displayed in the view such thatit covers a portion or whole of a specific body part of the characterobject with which it is associated.

The decorative object is an object that can be attached to a characterobject, for example, an accessory (such as a headband, a necklace, anearring, etc.), clothes (such as a T-shirt), a costume, and any otherobject which can be attached to the character object. The object data23B corresponding to the decorative object may include attachmentposition information indicating which body part of the character objectthe decorative object is associated with. The attachment positioninformation of a decorative object may indicate to which body part ofthe character object the decorative object is attached. For example,when the decorative object is a headband, the attachment positioninformation of the decorative object may indicate that the decorativeobject should be attached to the “head” of the character object. Whenthe decorative object is a T-shirt, the attachment position informationof the decorative object may indicate that the decorative object shouldbe attached to the “torso” of the character object.

The decorative object is added to the list stored on the storage 23 orany other storage by the server 20, and then it is attached to thecharacter object when it is selected by the actor or the supporter. Aviewing user feels close to the character object when the characterobject wears the decorative gift object the viewing user sent. On theother hand, when the viewing user requests display of a decorativeobject but the actor or the supporter does not select a correspondingdecorative object, the decorative object is not attached to thecharacter object. When the actor is to select a decorative object, theactor refers to the list displayed on the sub display 43 and selects thedecorative object by using a pointing device such as the controller 40or providing an instruction to the supporter. The supporter refers tothe list displayed on the supporter computer 45 and selects thedecorative object using the input interface 50.

The normal object is a gift object other than the decorative object,designed to be placed in the virtual space as a physical object.Examples of the normal object include objects representing a stuffed toyor a bouquet. The normal object may be displayed so as to overlap withthe character object, but unlike the decorative object, it is notdisplayed in association with a specific body part of the characterobject. For example, it is possible that the actor can change theposition of the normal object, whereas the actor cannot change theposition of the effect object. It is also possible that the duration ofdisplay of the normal object is different from that of the effectobject.

A duration of display may be set for each type of gift objects. Forexample, the duration of display of the decorative object may be setlonger than that of the effect object or the normal object. For example,the duration of display may be set to 60 seconds for the decorativeobject, 5 seconds for the effect object, and 10 seconds for the normalobject.

As shown in FIG. 5, the user management data 23C contains, for eachcontent of a character object, information on viewing users who haveviewed the video. The contents are basically distributed for multipletimes. For example, “distributor A” distributes “content P” featuring“character P” for multiple times. Also, “distributor B” distributes“content Q” featuring “character Q” for multiple times. The contents donot necessarily differ for each character object. For example, videosfeaturing the same character object but not related to each other mayconstitute separate contents. The user management data 23C containsrecords of activities of viewing users after the start of initialdistribution of contents. The activities are behaviors of the viewingusers on the contents and may include at least one of: a history ofviewing the contents, a status of posting messages to the contents, astatus of providing gift objects to the contents, and a status ofpayment to the contents. If “viewing user C” has viewed “content P” and“content Q” the user management data 23C records the activities of“viewing user C” on “content P” and the activities of “viewing user C”on “content Q.”

The user management data 23C contains, for each viewing user of thecontents, identification information (user ID) of the viewing user, agroup to which the viewing user belongs, a room visit status of theviewing user, a ranking of the viewing user, and a payment status of theviewing user.

In FIG. 5, the “Group” corresponding to a content indicates the group towhich a viewing user belongs for the content. The number of the groupsof the viewing users is any plural number. Each viewing user is assignedto a group based on the conditions related to his/her activities for thecontents he/she has viewed. The conditions for the assignment to thegroups may be set previously by the distributor or the administrator ofthe studio unit 30. For example, based on the conditions related toactivities for a content, the viewing users may be assigned to a groupof viewing users who are avid fans of the content or a group of viewingusers less related to the content such as those viewing the content forthe first time.

The conditions for assignment to the groups may include at least one of:the number of times of viewing the content, the number of times ofposting a post message to the content, the number of times of sending agift object to the content, and the amount of money paid to the content.For example, if the condition for assignment to the groups is the numberof times of viewing, viewing users who have viewed “content P” for notless than a predetermined number of times (e.g., ten times) are assignedto a first group (e.g., “VIP group”), and viewing users who have viewed“content P” for less than the predetermined number of times are assignedto a second group. Alternatively, if the condition for assignment to thegroups is the number of times of posting a message, viewing users whohave posted a message for not less than a predetermined number of times(e.g., 20 times) are assigned to a first group, and viewing users whohave posted a message for less than the predetermined number of timesare assigned to a second group. If the condition for assignment to thegroups is the number of times of providing a gift object, viewing userswho have provided a gift object for not less than a predetermined numberof times (e.g., five times) are assigned to a first group, and viewingusers who have provided a gift object for less than the predeterminednumber of times are assigned to a second group. If the condition forassignment to the groups is the amount of money paid, viewing users whohave paid not less than a predetermined amount of money are assigned toa first group, and viewing users who have paid less than a predeterminedamount of money are assigned to a second group. The viewing users may begrouped with combined conditions of: the number of times of viewing thecontent, the number of times of posting a post message to the content,the number of times of sending a gift object to the content, the amountof money paid to the content, and any other conditions.

The conditions for grouping may be other than those described above.Examples of such conditions include a use status (such as the frequencyof use, the number of years of use, and the last date and time of use)and user attributes (such as the age, sex, occupation, place ofresidence, and language). Further, these conditions may be combined forgrouping.

In FIG. 5, the “Room Visit Status” associated with a content is dataindicating whether or not a viewing user has logged in to the videodistribution system 1 and is viewing the content. When the content iscurrently being delivered to the client device 10 of the viewing user,the field “Room Visit Status” contains data indicating “visiting.” Whenthe content is currently not being delivered to the client device 10,the field “Room Visit Status” contains data indicating “left.” Inaddition, when the content is delivered to the client device 10, andthen the viewing user suspends viewing the video and does not resume itfor a predetermined period of time, the data indicating “Left” may bestored. If the content of a distributed video has not yet been viewed bythe viewing user, the data indicating “Unviewed” may be stored insteadof the data indicating “Left.”

In FIG. 5, the “Ranking” associated with a content is data indicatingthe ranking of a viewing user for the content. The method of ranking maybe determined by the administrator of the studio unit 30 or thedistributor. For example, the ranking may be determined in accordancewith the number of times of viewing the content, the number of times ofposting a post message to the content, the number of times of sending agift object to the content, and the amount of money paid to the content.The population for the ranking may be all the viewing users of thecontent or may be at least one of a plurality of groups (e.g., the firstgroup).

In FIG. 5, the “Payment Status” associated with a content is dataindicating the payment status of a viewing user for the content. Thedata of “Payment Status” include “Paid” and “Not Paid.” The paymentstatus may be replaced with the amount of paid money.

As shown in FIG. 6, the possession list 23D is a list of objects(possessed objects) possessed by a viewing user. The possession list 23Dmay include identification information of possessed objects (object IDs)in association with the viewing user. The possessed objects may beobjects different for each content or objects common to differentcontents, or the possessed objects may include both these objects. Forthe objects different for each content, the possession list 23D of theviewing user stores identification information of an object for eachcontent. The possession list 23D is stored continuously for a pluralityof distributed videos. The possession list 23D may be newly generated orreset each time a content for one distribution is distributed.

As shown in FIG. 7, the candidate list 23E is a list of gift objectsassociated with a video. The candidate list 23E is preferably generatedor reset each time a video is distributed for one view. When the viewinguser sends a display request for a gift object to the video, informationon the gift object is added to the candidate list 23E. The candidatelist 23E may include identification information (user ID) of the viewinguser who sent the gift object, identification information (object ID) ofthe gift object, the type of the gift object (decorative object, effectobject, normal object), the sending date and time of the gift object,and the selection status of the gift object. If the gift object is adecorative object, the selection status of the gift object indicateswhether or not the decorative object has been selected by the actor orthe supporter. When the decorative object has been selected, the field“Selection Status” stores data indicating “Selected,” and when thedecorative object has not been selected, the field “Selection Status”stores data indicating “Unselected.” If the gift object is not adecorative object, the field “Selection Status” stores data indicating“Unselected” or some data other than “Unselected” and “Selected.”

As shown in FIG. 8, the post message 23F may include information onmessages posted to videos by the viewing users. The post message 23F mayinclude, for each content, identification information (user ID) of theviewing user, the group to which the viewing user belongs, the messagetext, and the posting date and time of the message. If the group towhich the viewing user belongs is recorded on the user management data23C, it may not be recorded on the post message 23F. The post message23F may be newly generated or reset each time a content for onedistribution is distributed.

Next, a description is given of a view referred to by the viewing user,the actor, and the supporter, along with the flow of distribution of thecontent. When the viewing application program is started by theoperation of the viewing user, the client device 10 requests a list ofcontents from the server 20. The server 20 sends to the client device 10information for displaying a list of contents.

As shown in FIG. 9, the display 16 of the client device 10 displays acontent list view 100. The content list view 100 contains a list ofcontents being distributed or to be distributed This list includes theimage 101 of the content, the distribution date and time of the content,the content name, and the name of a character appearing in the content,and so on.

The viewing user selects one content from the list. The client device 10sends to the server 20 a delivery request for the selected content. Whenreceiving the delivery request for the content from the client device10, the server 20 delivers the content corresponding to the deliveryrequest to the client device 10. The client device 10 receives thecontent and displays the video.

As shown in FIG. 10, the display 16 of the client device 10 displays avideo playing view 105. The video playing view 105 contains a virtualspace viewed from a camera viewpoint set at the viewpoint sensor 36, anda character object 110 reflecting the motions of the actor. The videoplaying view 105 also contains a gift selection portion 112 and aposting selection portion 113.

When the viewing user selects the gift selection portion 112 of thevideo playing view 105, the client device 10 retrieves the possessionlist 23D from the server 20. Alternatively, the client device 10retrieves the possession list 23D that was previously retrieved from theserver 20 and stored on the storage 13. Further, the client device 10generates a list of gift objects possessed by the viewing user based onthe possession list 23D and displays the generated list.

When the viewing user selects a normal object from the list of giftobjects, the client device 10 sends a display request for the normalobject to the server 20. The display request includes identificationinformation (object ID) of the selected normal object and identificationinformation of the user (user ID). Since a display request for a giftobject needs to be sent with identification information of the user, theviewing user is required to log in to the video distribution system 1 ora separate user authentication system cooperating with the videodistribution system 1 before sending the display request.

In response to the display request for the normal object, the server 20displays the normal object 111 in the video distributed to the viewinguser who sent the display request and other viewing users. In theillustrated embodiment, a plurality of types of normal objects 111A to111C are displayed. The normal object 111A is a teddy bear object, andit is placed at a predetermined position in the virtual space. Thenormal object 111A may be movable within the virtual space whencontacted by the character object 110, or specifically, for example,when the character object 110 performs a motion to grasp the normalobject 111A. The normal object 111B is a hanging scroll object, and itis displayed at a predetermined position in the virtual space. Thenormal object 111C is able to move within the virtual space. Aftermoving within the virtual space, the normal object 111C may eventuallystop at a position reached or disappear from the view.

As shown in FIG. 11, when the viewing user sends a display request foran effect object, the effect object is displayed in the video playingview 105, as with the display request for a normal object. In theillustrated embodiment, the effect object 115 corresponding to confettiis displayed.

As shown in FIG. 12, when the viewing user sends a display request for adecorative object, the decorative object 120 is displayed in the videoplaying view 105, as with the display request for a normal object. Thedecorative object 120 is displayed in the video playing view 105 inassociation with a specific body part of the character object. Thedecorative object 120 may be displayed such that it contacts with thespecific body part of the character object. For example, the decorativeobject 120 representing ears of a cat (cat ears) is associated with themiddle of the head of the character object, and therefore, it isattached to the middle of the head of the character object 110. When thecharacter object 110 moves, the decorative object 120, which remainsassociated with the character object 110, also moves to follow the headas if it is attached to the character object 110.

If a video includes a period of time during which post messages areallowed to be posted, when any one of the viewing users viewing thevideo selects the posting selection portion 113, a message input view isdisplayed separately from the video playing view 105 or in the videoplaying view 105. When a message is input to the input view, theinformation of the input message is sent to the server 20 as a postmessage.

When receiving the post message within the period of time during whichposting is allowed, the server 20 sends video data containing the postmessage to the client device 10. The client device 10 causes the postmessage 130 to be displayed in the video playing view 105. The postmessage 130 is displayed in a predetermined message display region 131.The post messages 130 are arranged in the order of the posting date andtime. For example, the newest post message 130 is displayed at theuppermost portion of the message display region 131. When the server 20receives a new post message, the post message 130 displayed at theuppermost portion is moved down, and the new post message is displayedat the uppermost portion. The post message having reached the lowermostportion of the message display region 131 is moved further down anddisappears when a new message is added at the uppermost portion.Alternatively, only one post message may be displayed in the messagedisplay region 131.

Next, a description is given of a view referred to by the actor in thestudio room R. FIG. 13 shows an example of a notification view 200displayed on the sub display 43 installed in the studio room R. Thenotification view 200 may contain notification 201 to the actor. Thedata for displaying the notification view 200 may be generated by thesupporter computer 45 either automatically or via input operation of thesupporter. In the illustrated embodiment, the notification 201 includesthe remaining time, the schedule, the information notified to the actor,and so on. The actor gives the performance, while referring to thenotification view 200. The notification 201 may be displayed on thescreen of the main display 41 such that it is superimposed on or placedadjacent to the animation of the character object.

FIG. 14 shows an example of a gift list view 210 displayed on the subdisplay 43. The gift list view 210 contains lists (object information)of gift objects for which a display request has been sent from theviewing users, and these gift objects are displayed in association withthe viewing users. The gift list view 210 contains a gift list 211including all the gift objects, an attached gift list 212 (first list),and an unattached gift list 213 (second list).

The gift list 211 may include the image of a gift object for which arequest has been sent from the viewing user during distribution of thevideo, the name of the gift object, identification information of theviewing user (account name or user ID of the user) who has sent thedisplay request for the gift object, and the attachment status. The giftlist 211 includes the decorative objects, and preferably also includesother types of gift objects. The identification information of theviewing user may be an avatar of the viewing user. In the illustratedembodiment, the gift list 211 includes a normal object, an effectobject, and decorative objects, that is, all the types of gift objects.

For a decorative object included in the gift list 211, the “AttachmentStatus” in the gift list 211 indicates whether or not a character objectwears the decorative object. When the character object wears thedecorative object, the attachment status is “Attached.” When thecharacter object no longer wears it, the attachment status is“Finished.” In a content being distributed, decorative objects no longerworn by the character object are provided with the status of “Finished.”The actor and the supporter can refer to the gift list 211 to confirmthe gift objects for which a request has been sent from the viewingusers in a period from the time at which the distribution of the videowas started to the present time. Since the gift list 211 contains theidentification information of the user in association with the giftobject, the actor can send a message (such as “Thank you, A”) via thevoice in the video to the viewing user who has sent a display requestfor the gift object. This enables the actor to enhance the communicationwith the viewing user, leading to increased satisfaction of the viewinguser.

The duration of display may be set for a decorative object. When theduration of display of the decorative object has elapsed, the decorativeobject shown in the gift list 211 is provided with the selection statusof “Finished” and grayed out or shown in some other way different thanthe decorative objects of which the duration of display has not elapsed.The decorative object disappears from the video when its duration ofdisplay has elapsed.

When different viewing users requested display of the same decorativeobject in a video being distributed and only one of the decorativeobjects is worn by the character object, this decorative object may beprovided with the attachment status of “Attached” and the other mayremain with the attachment status of “Unattached.” Alternatively, boththese decorative objects may be provided with the attachment status of“Attached.”

The attached gift list 212 includes decorative gift objects selected bythe actor or the supporter and currently worn by the character object.The items of the attached gift list 212 are the same as those of thegift list 211. Among the decorative objects currently attached, one forwhich display was requested at the latest date and time, or the newestone is displayed at the uppermost portion of the attached gift list 212,and the others are arranged downward therefrom in the descending orderof the date and time at which the display was requested therefor. Theactor and the supporter refer to the attached gift list 212 to confirmthe decorative objects worn by the character object. Therefore, theactor can send a message representing, for example, his/her feelingabout the decorative object attached to the character object (such as “Ilove these cat ears”) to the viewing user via the voice in the video.This enables the actor to enhance the communication with the viewinguser with the medium of the decorative object, leading to increasedsatisfaction of the viewing user who offered the decorative object andurging the viewing users other than this viewing user to offer adecorative object.

When the duration of display of the decorative object has elapsed, thedecorative object is erased from the attached gift list 212, or it isgrayed out or shown in some other way different than the decorativeobjects of which the duration of display has not elapsed.

The unattached gift list 213 includes decorative gift objects for whichdisplay is requested by the viewing user and which is thereafter notselected by the actor or the supporter. The items of the unattached giftlist 213 are the same as those of the gift list 211. The decorativeobject for which display was requested at the earliest date and time, orthe decorative object remaining unattached for the longest time isdisplayed at the uppermost portion of the unattached gift list 213, andthe others are arranged downward therefrom in the ascending order of thedate and time at which the display was requested therefor. In this way,since the list of unattached decorative gift objects is displayed, theactor or the supporter can confirm the decorative objects that wereoffered by the viewing users and have not been selected. Since the actorcan readily refer to the unselected decorative objects, the actor canefficiently select as many decorative objects as possible. Further,since decorative objects for which display was requested at an earlierdate and time are displayed in an upper portion, it can be inhibited toomit a decorative object from selection.

Among the decorative gift objects, one having a limited duration ofdisplay set therefor disappears from the video playing view 105 when theduration of display has elapsed after it is attached to the characterobject. The decorative object that has disappeared is erased from theattached gift list 212. In the gift list 211, the selection status ofthe decorative object that has disappeared from the video playing view105 is updated to “Finished!” The duration of display of the gift objectmay be included in the gift list 211, the attached gift list 212, andthe unattached gift list 213.

When any one of the decorative gifts included in the unattached giftlist 213 is selected by the actor or the supporter, the server 20 erasesinformation on the selected decorative object from the unattached giftlist 213 and adds it to the attached gift list 212. Also, the selectionstatus of the selected decorative gift object in the gift list 211 isupdated from “Unselected” to “Attached.”

In this way, since the gift list view 210 contains the attached giftlist 212 and the unattached gift list 213 that are separate from eachother, the actor or the supporter can confirm the selected decorativeobjects and the unselected decorative objects and thus can select thedecorative objects efficiently even when the viewing users offer a largenumber of decorative objects. The chances for communication with theviewing users can be increased by selecting as many decorative objectsas possible to be attached to the character object within a limiteddistribution time. This increases satisfaction of the viewing users withthe distributed video.

FIG. 15 shows an example of a message view 220 displayed on the subdisplay 43. The message view 20 may be displayed on the same sub display43 as the gift list view 210, or it may be displayed on a different subdisplay 43 than the gift list view 210 when there are a plurality of subdisplays 43. The message view 220 contains a first message displayregion 221, a second message display region 222, and a room visit log223.

The first message display region 221 includes a list of post messages23F associated with the content being distributed. These post messagesare read out from the storage 23 by the computer processor 21 of theserver 20. The post messages 23F are arranged in the order of theposting date and time. The post message having the latest posting dateand time, or the newest post message may be displayed at the uppermostportion of the list. When a new post message is posted, this postmessage is added at the uppermost portion of the first message displayregion 221, and the post messages already displayed in the first messagedisplay region 221 are moved downward. It is also possible that the postmessages are moved in the opposite direction. Specifically, the newestpost message is added at the lowermost portion of the list, and the postmessages already displayed in the first message display region 221 aremoved upward. The list may contain, for each post message,identification information of the viewing user (such as the account nameof the viewing user), a posting date and time, and a content of themessage. The identification information of the viewing user may be anavatar of the viewing user.

The second message display region 222 includes a list of post messagesfrom the viewing users who belong to the first group (VIP group). Thesepost messages are extracted by the computer processor 21 of the server20 from the post messages 23F stored on the storage 23. In theillustrated embodiment, the post messages displayed in the secondmessage display region 222 are also displayed in the first messagedisplay region 221. Alternatively, it is also possible that the postmessages displayed in the second message display region 222 are notdisplayed in the first message display region 221. The list displayed inthe second message display region 222 may contain a ranking of theviewing user, in addition to identification information of the viewinguser (such as the account name of the viewing user), a posting date andtime, and a content of the message. The identification information ofthe viewing user may be an avatar of the viewing user.

The room visit log 223 is a list of the viewing users viewing thecontent being distributed. The computer processor 21 of the server 20extracts the viewing users having the room visit status of “Visiting”with reference to the user management data 23C associated with thecontent being distributed. The room visit log 223 may containidentification information of the viewing user currently viewing thecontent, a room visit date and time, a room visit status, and a group.The identification information of the viewing user may be an avatar ofthe viewing user.

The actor can refer to the room visit log 223 to send a message such as“Hello, N” via the voice in the video to a viewing user who startedviewing. In this way, since the actor can communicate with the viewinguser who started viewing, the viewing user becomes fond of the characterand is more satisfied with the content.

The actor can refer to the first message display region 221 and thesecond message display region 222 to deal with the post messages fromthe viewing users. Since the first message display region 221 isseparated from the second message display region 222 for the viewingusers of the first group, it is possible to readily grasp the tendencyof the users who sent the post messages. For example, the actor can dealwith the post messages from the viewing users of the first group bypriority. This particularly increases the satisfaction of the viewingusers of the first group with the content. Alternatively, when the actordeals with the post messages from the viewing users of the second groupby priority, it is possible to increase particularly the satisfaction ofthe viewing users of the second group with the content.

Next, a video distribution process in one embodiment will be describedwith reference to FIG. 16, In the video distribution process, it isassumed that the actor is giving a performance in the studio room R. Instep S11, the motion data processing unit 21A of the server 20 generatesthe body motion data, which is a digital representation of the bodymotions of the actor, and the face motion data, which is a digitalrepresentation of the facial motions (expression) of the actor.

Subsequently, in step S12, the video generation unit 21B generates avideo. The video generation unit 21B applies the body motion data andthe face motion data of the actor to the model data 23A for the actor,thereby generating animation of the character object that moves insynchronization with the motions of the body and facial expression ofthe actor. The video generation unit 21B further generates a video usingthis animation. The voices of the actor are applied to the video.

Subsequently, in step S13, the video distribution unit 21C distributesthe video generated in step S12 as a content. The content is distributedto the client devices 10 over the network 11. The content may bedelivered to the supporter computer 45 and/or may be projected on themain display 41 in the studio room R. A content for one distribution hasa predetermined length of time and is distributed continuously for thislength of time. The length of time of the video may be set to, forexample, 30 seconds, 1 minute, 5 minutes, 10 minutes, 30 minutes, 60minutes, 120 minutes, and any other length of time.

Subsequently, in step S14, it is determined whether or not an endingcondition for ending the distribution of the video is met. The endingcondition is, for example, that the distribution ending time has beenreached, that the supporter computer 45 has issued an instruction to endthe distribution, or any other conditions. If it is determined that theending condition is not met, steps S11 to S13 of the process arerepeated, and the video including the animation synchronized with themotions of the actor is continuously distributed. If it is determinedthat the ending condition is met for the video, the distribution processof the video is ended.

Next, a reception process of a gift object will be described withreference to FIG. 17. When receiving a display request for a gift objectsent from a client device 10, the user request processing unit 21D ofthe server 20 receives the gift object (step S20). The display requestmay include the identification information of the viewing user, theidentification information of the gift object, the sending date andtime, and so on.

The user request processing unit 21D adds the information on thereceived gift object to the candidate list 23E, and adds the informationon this gift object to the gift list 211 of the gift list view 210 (stepS21).

Subsequently, the user request processing unit 21D determines whether ornot the added gift object is a decorative object (step S22). If the giftobject is not a decorative object, that is, the gift object is an effectobject or a normal object (No in step S22), the reception process of thegift object is ended.

If the gift object is a decorative object (Yes in step S22), the userrequest processing unit 21D adds the gift object to the unattached giftlist 213 (step S23).

Next, a selection process of a decorative object will be described withreference to FIG. 18. If it is determined that a decorative object isselected by operation of the actor or the supporter (Yes in step S30),the user request processing unit 21D erases the selected decorativeobject from the unattached gift list 213 and adds it to the attachedgift list 212 (step S31). The video generation unit 21B displays theselected decorative object at a predetermined body part of the characterobject associated with the decorative object (step S32).

Further, it is determined whether or not the duration of display of thedecorative object displayed in the video has elapsed (step S33). If theduration of display of the decorative object has not elapsed (No in stepS33), the decorative object is displayed continuously. On the otherhand, if the duration of display of the decorative object has elapsed(Yes in step S33), the attached gift list 212 is updated to change thestatus of the decorative object of which the duration of display haselapsed. Specifically, the decorative object of which the duration ofdisplay has elapsed is erased from the attached gift list 212, or it isgrayed out or shown in some other way different than the decorativeobjects of which the duration of display has not elapsed.

Next, a description is given of a display process of a post message withreference to FIG. 19. When receiving identification information and apost message of a viewing user, the user request processing unit 21Ddetermines whether or not the viewing user who sent the post messagebelongs to the first group (step S40). The user information managementunit 21E groups the viewing users at a predetermined timing.

If it is determined that the viewing user who sent the received postmessage belongs to the first group (Yes in step S40), the user requestprocessing unit 21D displays the post message in the first messagedisplay region 221 and the second message display region 222 (step S41).On the other hand, if it is determined that the viewing user who sentthe received post message belongs to the second group (Yes in step S40),the user request processing unit 21D displays the post message in onlythe second message display region 222 (step S42).

As described above, the embodiment provides the following advantageouseffects. (1) The video distribution system receives a display requestfor an object from a viewing user. A list of objects for which a displayrequest has been received is displayed on the sub display 43 disposed atsuch a position as can be seen by the actor. The actor, who is giving aperformance, can refer to the list displayed on the sub display 43 toquickly confirm the objects for which display is requested and theviewing users who requested display of these objects. Further, theactor, who quickly confirms the objects and the viewing users, can sendmessages to the viewing users who requested display at suitable timings.Accordingly, the actor can increase the chances for communication withthe viewing users.

(2) Objects are selected from the gift list view 210 by the actor or anyother users except the viewing users, and the selected objects aredisplayed in the video. Since the gift list view 210 includes objectsfor which display is requested, the actor or the like can select theobjects efficiently. Therefore, the advantageous effects of the list canbe enhanced.

(3) In the gift list view 210, selected objects and unselected objectsare divided into separate lists. Therefore, for example, it isfacilitated to select an object from the list of unselected objects.This supports the performance of the actor.

(4) Only the decorative objects are displayed in the attached gift list212 and the unattached gift list 213. In other words, only the objectsthat need to be selected by a user other than the viewing users aredisplayed in the first list and the second list. Therefore, the actorcan readily select a desired decorative object.

(5) The post messages from the viewing users of the first group aredisplayed in a different region than the post messages from the viewingusers of the other group. Therefore, the actor can deal with the postmessages from the viewing users of the first group by priority.Accordingly, the actor can increase the chances for communication withparticular viewing users.

The foregoing embodiment can be modified as described below. The aboveembodiment and the following modifications can be implemented incombination to the extent where they are technically consistent witheach other. In the above embodiment, the gift list view 210 contains theattached git list 212 and the unattached gift list 213 and is displayedon the sub display 43. Alternatively or additionally, at least one of alist of gifts from the viewing users of the first group and a list ofgifts from the viewing users of the second group may be displayed on thesub display 43. Further, the list of gifts from the viewing users of thefirst group may be divided into an attached gift list and an unattachedgift list. Likewise, the list of gifts from the viewing users of thesecond group may be divided into an attached gift list and an unattachedgift list. In this way, the decorative gift objects can be displayeddistinguishably in terms of the group of the viewing users in additionto whether the objects are attached or unattached. Therefore, it ispossible to communicate with viewing users in accordance with the groupthereof.

In the above embodiment, the gift list view 210 and the message view 220are separate from each other. Alternatively, two or more of the attachedgift list 212, the unattached gift list 213, the first message displayregion 221, the second message display region 222, and the room visitlog 223 may be displayed adjacent to each other in one screen. It isalso possible that the actor or the supporter can freely arrange in onescreen any of the gift list 211, the attached gift list 212, theunattached gift list 213, the first message display region 221, thesecond message display region 222, and the room visit log 223.

In the above embodiment, one sub display 43 may display the gift listview 210 and the message view 220 alternately. These views may beswitched upon reception of a display request for a gift object or a postmessage. For example, when a display request for a gift object isreceived, the gift list view 210 is displayed for a predeterminedperiod, and when a post message is received, the message view 220 isdisplayed for a predetermined period.

In the above embodiment, the gift list view 210 and the message view 220are displayed on the sub display 43. Alternatively, at least one of thegift list view 210 and the message view 220 may be displayed on the maindisplay 41.

In the above embodiment, a decorative object disappears when apredetermined duration of display has elapsed after it is attached to acharacter object. Alternatively, a decorative object may disappear byoperation of the actor or the supporter.

In the above embodiment, the gift list view 210 and the message view 220are displayed on the sub display 43. Alternatively or additionally,these views may be displayed on a head-mounted display attached to theactor. The head-mounted display may be either transparent ornontransparent. The head-mounted display may display videos includinganimation of a character, or it may display, instead of such videos,only at least one of the gift list view 210 and the message view 220.

In the above embodiment, the viewing users are divided into a pluralityof groups in accordance with the number of times of viewing the content,the number of times of posting a post message to the content, the numberof times of sending a gift object to the content, and the amount ofmoney paid to the content. Alternatively, the viewing users may bedivided into the following two groups: the group of viewing users whobehave improperly to the distributed video (e.g., a second group), andthe group of the other viewing users (e.g., a first group). Sinceinformation that identifies a viewing user, such as the real name or theaddress of the viewing user, is not disclosed to the distributor orother viewing users, except for necessary information such as theaccount name, the user ID, or the avatar, a malicious viewing user maybehave improperly in activities such as posting a message to the videoand requesting display of a gift object. To prevent a trouble, thedistributor needs to quickly perceive such viewing users “visiting theroom” (logging in), posting a message, and so on. The condition forassignment to groups is, for example, that a viewing user has previouslysent an improper message to the video or the number of times of his/herdoing so has reached a threshold value. The way to determine that amessage is improper is, for example, that the server 20 analyzes thecontent of the message and determines that the message is improper whenit determines that a word contained in the message is improper. Anotherway is that the distributor or the administrator visually check themessages posted to the video to determine whether or not the message isimproper, and still another way is to determine that a message isimproper when any other viewing user reports it. When the distributor orthe administrator determines whether or not a message is improper, thedistributor or the administrator operates the supporter computer 45 orthe like to assign the viewing user who sent the message to, forexample, “a second group.” Alternatively, the distributor or theadministrator operates the supporter computer 45 or the like to updatethe number of times that the viewing user who sent the message made animproper remark. The server 20 receives such information from thesupporter computer 45 or the like to update the user management data.Such grouping may be managed for each content as in the aboveembodiment, or it may be managed generally for the entire platform fordistributing the video. The room visit log 223 may include the group towhich a viewing user belongs, in addition to the identificationinformation of the viewing user. When a viewing user who belongs to thesecond group and needs to be monitored “visits the room”, theidentification information of this viewing user is displayed in the roomvisit log 223 (FIG. 15), and therefore, the distributor can force theviewing user to leave the room. Alternatively, when the viewing user“visits the room,” the server 20 may automatically force the viewinguser to leave the room. Instead of forcing the viewing user to leave theroom, the distributor may pay attention to prevent the viewing user whoneeds to be monitored from bothering other viewing users after the roomvisit log 223 indicates that the viewing user who needs to be monitoredvisited the room.

In one embodiment, a plurality of infrared LEDs are mounted on each ofthe wearable sensors 33 attached to the actor, and light from theinfrared LEDs are sensed by infrared cameras provided on the floorand/or wall of the studio room R to detect the position and theorientation of each of the wearable sensors 33. Visible light LEDs maybe used instead of the infrared LEDs, and in this case light from thevisible light LEDs may be sensed by visible light cameras to detect theposition and the orientation of each of the wearable sensors 33. Asdescribed above, a light emitting unit (for example, the infrared LED orvisible light LED) may be provided on each of the plurality of wearablesensors 33 attached to the actor, and a light receiving unit (forexample, the infrared camera or visible light camera) provided in thestudio room senses the light from the light emitting unit to detect theposition and the orientation of each of the wearable sensors 33. In oneembodiment, a plurality of markers may be used instead of the wearablesensors 33. The markers are provided on each actor. Alternatively, themarkers may be provided on the costume of the actor (e.g., an uppergarment, a lower garment, or a suit including these garments combined).At least one of the position and the orientation of each marker can bedetected by capturing images of the actor having the markers attachedthereto to generate captured image data and performing image processingon the captured image data. It is also possible that the wearable sensor33, sensors that can sense at least one of position and orientation suchas acceleration sensors or gyroscopes, electrostatic sensors, or anyother sensors that can send signals to and receive signals from externalsensors 32 and/or the server 20 are provided on the costume of the actor(e.g., an upper garment, a lower garment, or a suit including thesegarments combined), and at least one of the position, the orientation,and the motion of each of the sensors based on the signals sent fromthese sensors. Further, the tracking system 31 may be capable of detectthe motions of the actor with only external sensors 32 and without usingthe wearable sensors 33.

The method according to the embodiment may be implemented by a programthat can be executed by a computer. The program may be stored on, forexample, a magnetic disc (such as Floppy™ disk and hard disk), anoptical disc (such as CD-ROM, DVD, and MO), a semiconductor memory (suchas ROM, RAM, and flash memory), or any other storage medium ornon-transitory computer-readable storage medium. The program may also betransmitted by a communication medium for distribution. The programstored on the storage medium may include a setting program forconstructing a software means (including not only an execution program,but also tables and data structures) to be executed by the computer. Thecomputer that realizes this device reads the program stored on thestorage medium and, if necessary, constructs the software means by thesetting program, and the software means controls the operation toperform the process described above. The term “storage medium” usedherein includes not only those for distribution, but also storage mediasuch as a magnetic disc and a semiconductor memory installed in thecomputer or connected over a network.

The computer processors 12, 21, and 46 do not necessarily performsoftware processing for all the processes they perform. For example, thecomputer processors 12, 21, and 46 may include a dedicated hardwarecircuit (e.g., application-specific integrated circuits (ASIC)) forhardware processing for at least a part of the processes they perform.Specifically, the computer processors 12, 21, and 46 may be formed as acircuitry including: 1) one or more processors that operate inaccordance with a computer program (software); 2) one or more dedicatedhardware circuits that perform at least a part of the various processes;or 3) a combination thereof. The processors include CPU and a memorysuch as RAM and ROM, and the memory stores program codes or instructionsconfigured to cause the CPU to perform processes. The memory, or acomputer-readable medium, encompasses any kind of available mediaaccessible via a general-purpose or dedicated computer.

1. A video distribution system for distributing data for causing clientdevices used by a plurality of viewing users to play a video includinganimation of a character object generated based on a motion of an actorto the client devices, the video distribution system comprising: one ormore computer processors, wherein the one or more computer processorsexecute computer-readable instructions to: receive, from a first viewinguser of the plurality of viewing users, a display request to display afirst object of a plurality of objects; and responsive to the displayrequest, display object information on one or more display devicesdisposed at a position viewable by the actor, the object informationincluding information on the first object and identification informationof the first viewing user.
 2. The video distribution system of claim 1,wherein in response to a selection of the first object by a selectinguser, cause the first object to be displayed in the video, wherein theselecting user differs from the first viewing user.
 3. The videodistribution system of claim 2, wherein the object information includesa first list and a second list, the first list including first objectinformation on the first object selected by the selecting user, thesecond list including second object information on one or moreunselected objects, each of the one or more unselected objects being anyone of the plurality of objects for which another display request wasreceived and which has not been selected by the selecting user.
 4. Thevideo distribution system of claim 3, wherein: the plurality of objectsincludes one or more decorative objects and one or more other objects,the one or more decorative objects each being displayed in the video inresponse to a selection by the selecting user each of the one or moreother objects being displayed in the video in response to receiving adisplay request for each of the one or more other objects, and the firstlist and the second list include information on the one or moredecorative objects only.
 5. The video distribution system of claim 1,wherein when behavior of one of the plurality of viewing users on thevideo satisfies a predetermined condition, the one or more computerprocessors are configured to: associate the one of the plurality ofviewing users with one of a plurality of groups.
 6. The videodistribution system of claim 1, wherein each of the plurality of viewingusers is associated with one of a plurality of groups, the one or morecomputer processors are configured to: receive post messages sent fromone or more of the plurality of viewing users to the video beingdistributed; and display a first message display region and a secondmessage display region, the first message display region beingconfigured to display post messages sent from one or more of theplurality of viewing users who belong to a predetermined group of theplurality of groups, and the second message display region beingconfigured to display post messages sent from one or more of theplurality of viewing users who do not belong to the predetermined group.7. A video distribution method performed by one or more computerprocessors executing computer-readable instructions to distribute datafor causing client devices used by a plurality of viewing users to playa video including animation of a character object generated based on amotion of an actor to the client devices, the video distribution methodcomprising: receiving, from a first viewing user of the plurality ofviewing users, a display request to display a first object of aplurality of objects; and responsive to the display request, causingobject information to be displayed on one or more display devicesdisposed at a position viewable by the actor, the object informationincluding information on the first object and identification informationof the first viewing user.
 8. A non-transitory computer-readable storagemedium storing a video distribution program for distributing data forcausing client devices used by a plurality of viewing users to play avideo including animation of a character object generated based on amotion of an actor to the client devices, the video distribution programcausing one or more computer processors to effectuate operationscomprising: receiving, from a first viewing user of the plurality ofviewing users, a display request to display a first object of aplurality of objects; and responsive to the display request, displayingobject information on one or more display devices disposed at a positionviewable by the actor, the object information including information onthe first object and identification information of the first viewinguser.